Method of removing a blocking artifact using quantization information in a filtering system

ABSTRACT

In one embodiment, the method includes obtaining quantization information related to at least one of a first block and an adjacent second block, comparing the quantization information with a predefined constant value, and performing a filtering process based on a result from the comparing step. The filtering process includes calculating a value of at least one pixel in the first block based on pixel values of at least two pixels in the first block, and calculating a value of at least one pixel in the second block based on pixel values of at least two pixels in the second block. The at least two pixels in the first block and the at least two pixels in the second block are successive pixels across a block boundary between the first block and the second block.

PRIORITY INFORMATION

This is a continuation of U.S. application Ser. No. 13/494,385 filed onJun. 12, 2012, which is a continuation of U.S. application Ser. No.11/905,542 filed Oct. 2, 2007, now U.S. Pat. No. 8,238,422; which is adivisional of U.S. application Ser. No. 11/097,394, filed Apr. 4, 2005,now issued U.S. Pat. No. 7,711,054, which is a divisional of U.S.application Ser. No. 10/170,999 filed Jun. 14, 2002, now issued as U.S.Pat. No. 7,272,186; which claims priority under 35 U.S.C. 119 on KoreanApplication No. 2001-33953 filed Jun. 15, 2001; the contents of all ofwhich are hereby incorporated by reference in their entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a moving picture compression technique,and more particularly, to a loop filtering method in a video coder.

2. Description of the Background Art

Generally, it has been proved that the “H.26L” system developed as anext generation moving picture compression technique is superior to the“H.263” and “MPEG4” systems in performance.

The “H.26L” system as the next generation moving picture compressionsystem differs from the conventional H.263 and MPEG4 in using 4×4 blockbased transformation & encoding, carrying out motion estimation &compensation of a transformation block size, and using a single variablelength coder (VLC).

Superior to the conventional motion picture standards in aspect ofperformance, the H.26L system has excessive calculation of encoder aswell as uses 4×4 block based transformation so that a blocking dominatesthan the conventional standards.

In order to remove the blocking, filters include a loop filtering systeminside the encoder and decoder, a post filter processed in a next stageof the decoder, and a system adding the loop filtering system to thepost filter.

When the blocking is removed by the loop filtering system in general, adifference between an input video and a motion video is affected so thatan encoding rate increases in a specific case. Yet, as the loopfiltering system is installed inside a video coder, complexity of theencoder increases.

As mentioned in the above explanation, the H.26L system according to therelated art requires excessive quantity of calculation despite thesuperiority in performance to the moving picture standards as well ashaving the blocking more dominant than that of the moving picturestandards due to the 4×4 block based transformation system.

Moreover, if the blocking is removed by the loop filtering systemaccording to the related art, the difference between the input video andthe motion video is affected so as to increase the encoding rate in aspecific case. Yet, the loop filtering system built inside the videocoder increases the complexity of the encoder.

SUMMARY OF THE INVENTION

The present invention relates to a method of removing a blockingphenomenon using properties of two blocks.

In one embodiment, the method includes obtaining a coded block patternof first and second blocks. The coded block pattern may be one of aplurality of coded block patterns, the plurality of coded block patternsmay include an intra-coded block, and the second block may be adjacentto the first block. Whether the first block and the second block havenon-zero transform coefficient information is determined if the blocktype of the first block and the second block is not intra-coded block. Anon-zero filter strength value is determined based on the obtained codedblock pattern of the first and second block and the determination ofwhether the first block and second blocks have non-zero transformcoefficient information, and the blocking phenomenon is removedaccording to the filter strength value.

In one embodiment, the method includes obtaining quantizationinformation related to at least one of a first block and an adjacentsecond block, comparing the quantization information with a predefinedconstant value, and performing a filtering process based on a resultfrom the comparing step. The filtering process includes calculating avalue of at least one pixel in the first block based on pixel values ofat least two pixels in the first block, and calculating a value of atleast one pixel in the second block based on pixel values of at leasttwo pixels in the second block. The at least two pixels in the firstblock and the at least two pixels in the second block are successivepixels across a block boundary between the first block and the secondblock.

It is to be understood that both the foregoing general description andthe following detailed description of the present invention areexemplary and explanatory and are intended to provide furtherexplanation of the invention as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the invention and are incorporated in and constitute apart of this application, illustrate embodiment(s) of the invention andtogether with the description serve to explain the principle of theinvention. In the drawings:

FIG. 1 illustrates a flowchart of a loop filtering process according toan embodiment of the present invention;

FIGS. 2A and 2B illustrate block diagrams indicating block locations inhorizontal and vertical directions according to an embodiment of thepresent invention, respectively;

FIG. 3 illustrates a diagram of pixel locations of adjacent blocksaccording to an embodiment of the present invention; and

FIG. 4 illustrates a table of LQP values for determining DQP(QP) valuesaccording to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE EXAMPLE EMBODIMENTS

Reference will now be made in detail to the example embodiments of thepresent invention, which are illustrated in the accompanying drawings.

Hereinafter, a loop filtering method in a video coder according to anexample embodiment of the present invention carries out a real-timeprocessing of removing a blocking and a ringing phenomenon in accordancewith a pixel location, a coded block pattern, a quantization step size,and a motion vector size in a digital video appliance using the H.26Lmoving picture compression technique, thereby enabling to improve avideo quality of a compression image demanding a low bit rate orhigh-speed processing.

First, explained in detail is a real-time processing technique ofremoving “blocking” and “ringing” by predicting an original video forcorrelation between pixels in accordance with a quantization variableQP, a coded block pattern, and a motion vector size available for adecoder(not shown in the drawing).

When an original video f is transmitted as a compressed form, a videorestructured in a decoder can be expressed by the following Formula 1-1.

[Formula 1-1]

g=f+n, where g, f, and n are a compression video rearranged in ascanning order, an original video, and a column vector of quantizationerror, respectively.

A relation between respective pixels can be expressed by the followingFormula 1-2 in order to process the Formula 1-1 by pixel unit.

[Formula 1-2]

g(i,j)=f(i,j)+n(i,j), where i and j indicate a location of a pixel invertical and horizontal directions.

An additional function can be defined as the following Formula 1-3 whenreliability of original pixels and 1-dimensional smoothing function foreach pixel are used together with a regularization. Firstly, ahorizontal direction is considered.

[Formula 1-3]

M{f(i,j)}=M_(p){f(i,j)}+M_(N){f(i,j)}, where M_(p) and M_(N) arefunctions representing reliability smoothing degrees of left and rightdirections for a pixel f(i,j), respectively.

And, such functions are defined as the following Formula 1-4, where theM_(P) and M_(N) are the functions considering f(i,j) & f(i,j−1) andf(i,j) & f(i,j+1), respectively.

[Formula 1-4]M _(P) {f(i,j)}=α_(P) {f(i,j)−f(i,j−1)}²+(1−α_(P)){g(i,j)−f(i,j)}²M _(N) {f(i,j)}=α_(N) {f(i,j)−f(i,j−1)}²+(1−α_(N)){g(i,j)−f(i,j)}²

Each of the first terms of right sides in Formula 1-4 indicatesirregularity with an adjacent pixel, i.e. a difference from an adjacentpixel, for the pixel f(i,j), and each of the second terms indicates areliability for the original pixel f(i,j). Moreover, α_(P) and α_(N)mean regularization parameters representing ratios between irregular andreliability terms defined in the functions, respectively. And, α_(P) andα_(N) are parameters adjusting a relative coherence between the twoterms in Formula 1-4.

With the above manner, the additional function is defined for each pixelof the moving picture, and the function in a vertical direction isdefined by varying the parameter i instead of the parameter j in Formula1-4.

Thereafter, if a differential is applied to the additional functions inFormula 1-4 defined for each pixel of the moving picture in order tofind a restored video from which the blocking and ringing are removed,the following Formulas 1-5 and 1-6 are attained.

$\begin{matrix}{\frac{{\partial M}\left\{ {f\left( {i,j} \right)} \right\}}{\partial{f\left( {i,j} \right)}} = {{\frac{{\partial M_{P}}\left\{ {f\left( {i,j} \right)} \right\}}{\partial{f\left( {i,j} \right)}} + \frac{{\partial M_{N}}\left\{ {f\left( {i,j} \right)} \right\}}{\partial{f\left( {i,j} \right)}}} = 0}} & \left\lbrack {{Formula}\mspace{14mu} 1\text{-}5} \right\rbrack\end{matrix}$

From Formula 1-5, the following Formula 1-6 is found.

$\begin{matrix}{{\frac{{\partial M_{P}}\left\{ {f\left( {i,j} \right)} \right\}}{\partial{f\left( {i,j} \right)}} = {{2{\partial_{P}\left\{ {{f\left( {i,j} \right)} - {f\left( {i,{j - 1}} \right)}} \right\}}} - {2\left( {1 - \alpha_{P}} \right)\left\{ {{g\left( {i,j} \right)} - {f\left( {i,j} \right)}} \right\}}}}{\frac{{\partial M_{N}}\left\{ {f\left( {i,j} \right)} \right\}}{\partial{f\left( {i,j} \right)}} = {{2{\partial_{N}\left\{ {{f\left( {i,j} \right)} - {f\left( {i,{j - 1}} \right)}} \right\}}} - {2\left( {1 - \alpha_{N}} \right)\left\{ {{g\left( {i,j} \right)} - {f\left( {i,j} \right)}} \right\}}}}} & \left\lbrack {{Formula}\mspace{14mu} 1\text{-}6} \right\rbrack\end{matrix}$

From Formulas 1-5 and 1-6, the pixel to be restored in a horizontaldirection can be found by the following Formula 1-7.

$\begin{matrix}{{f\left( {i,j} \right)} = \frac{\mspace{31mu}{{\left( {2 - \alpha_{P} - \alpha_{N}} \right){g\left( {i,j} \right)}} + {\alpha_{P}{f\left( {i,{j - 1}} \right)}} + {\alpha_{N}{f\left( {i,{j + 1}} \right)}}}}{2}} & \left\lbrack {{Formula}\mspace{14mu} 1\text{-}7} \right\rbrack\end{matrix}$

As the regularization parameters in Formula 1-7 are between “0” and “1”,Formula 1-7 can be defined by the following Formula 1-8.

$\begin{matrix}{{f\left( {i,j} \right)} = \frac{\mspace{31mu}{{\left( {2 - \alpha_{P} - \alpha_{N}} \right){g\left( {i,j} \right)}} + {\alpha_{P}{g\left( {i,{j - 1}} \right)}} + {\alpha_{N}{g\left( {i,{j + 1}} \right)}}}}{2}} & \left\lbrack {{Formula}\mspace{14mu} 1\text{-}8} \right\rbrack\end{matrix}$

Looking into Formula 1-8, a blocking-removed video of the compressionvideo is determined by the two, left and right pixel values and theregularization parameters by taking the i_(th) and j_(th) videos asreferences.

As the two, left and right pixel values can be used for encoding anddecoding, two regularization parameter values are set up for attainingthe restored video. For this, “set theoretic” is applied to therestoration by pixel unit so as to represent regularization coefficientsby the following Formula 1-9, where “set theoretic” is a theory forcontrolling the parameters α_(P) and α_(N) in Formula 1-4.

$\begin{matrix}{{\frac{\alpha_{P}}{1 - \alpha_{P}} = {\frac{\left\{ {{f\left( {i,j} \right)} - {g\left( {i,j} \right)}} \right\}^{2}}{\left\{ {{f\left( {i,j} \right)} - {f\left( {i,{j - 1}} \right)}} \right\}^{2}} \approx \frac{K_{P}{QP}^{2}}{\left\{ {{g\left( {i,j} \right)} - {g\left( {i,{j - 1}} \right)}} \right\}^{2}}}},{\frac{\alpha_{N}}{1 - \alpha_{N}} = {\frac{\left\{ {{f\left( {i,j} \right)} - {g\left( {i,j} \right)}} \right\}^{2}}{\left\{ {{f\left( {i,j} \right)} - {f\left( {i,{j + 1}} \right)}} \right\}^{2}} \approx \frac{K_{N}{QP}^{2}}{\left\{ {{g\left( {i,j} \right)} - {g\left( {i,{j + 1}} \right)}} \right\}^{2}}}}} & \left\lbrack {{Formula}\mspace{14mu} 1\text{-}9} \right\rbrack\end{matrix}$

K_(P) and K_(N) in Formula 1-9 are determined statistically as adaptiveparameters depending on a location of a pixel since the irregularitiesbetween pixels at a block boundary and an inner block side are differentfrom each other. And, “QP” is a quantization parameter of a macro blockto which the pixel f(i,j) belongs.

Even though the restored video from which blocking is removed usingFormulas 1-8 and 1-9, the value of the regularization parameter requiresa floating-point operation so as to become a problem in aspect ofcalculation quantity of the filter to remove the “blocking”.

Therefore, Formulas 1-8 and 1-9 are transformed into the followingFormulas 1-10 and 1-11 for integer operation.

$\begin{matrix}{{f\left( {i,j} \right)} = \frac{\mspace{34mu}{{\left( {2^{9} - {\alpha_{P} \times 2^{8}} - {\alpha_{N} \times 2^{8}}} \right){g\left( {i,j} \right)}} + {\alpha_{P}2^{8} \times {g\left( {i,{j - 1}} \right)}} + {\alpha_{N} \times 2^{8} \times {g\left( {i,{j + 1}} \right)}}}}{2 \times 2^{8}}} & \left\lbrack {{Formula}\mspace{14mu} 1\text{-}10} \right\rbrack \\{{\beta_{P} = {{\alpha_{P} \times 2^{8}} = \frac{2^{8} \times K_{P}{QP}^{2}}{{K_{P}{QP}^{2}} + \left\{ {{g\left( {i,j} \right)} - {g\left( {i,{j - 1}} \right\}}^{2}} \right.}}},{\beta_{N} = {{\alpha_{N} \times 2^{8}} = \frac{2^{8} \times K_{N}{QP}^{2}}{{K_{N}{QP}^{2}} + \left\{ {{g\left( {i,j} \right)} - {g\left( {i,{j - 1}} \right\}}^{2}} \right.}}}} & \left\lbrack {{Formula}\mspace{14mu} 1\text{-}11} \right\rbrack\end{matrix}$

In this case, β_(P) and β_(N), as defined in Formula 1-11, are variablesattained by multiplying α_(P) and α_(N) by 2⁸, respectively. Namely,β_(P) and β_(N) are parameters for making α_(P) and α_(N) into integers.

In an algorithm according to the present invention, the regularizationparameters as integer forms using Formula 1-11 are generated, and thenstored as a look-up table form to be used.

Hence, Formula 1-10 for blocking filtering can be found by knowing thepixel f(i,j) and locations and quantization parameter values of twopixels adjacent to the pixel f(i,j).

Meanwhile, in the loop filtering process of the present invention, avideo of which blocking in a direction of a horizontal axis is removedby loop-filtering a block of a moving picture is attained, and then thevideo is loop-filtered in a direction of a vertical axis so as toprovide the video of which blocking in a direction of a vertical axis isremoved.

FIG. 1 illustrates a flowchart of a loop filtering process according toan embodiment of the present invention.

Referring to FIG. 1, a loop filtering method according to an embodimentof the present invention includes the steps of determining a strength ofa block to be processed in accordance with a coded pattern and aquantization step size of each 4×4 block when a block of a movingpicture is loop-filtered in directions of its horizontal and verticalaxes(S11), carrying out a loop filtering in accordance with thedetermined strength of the corresponding block(S12), and carrying out anadditional loop filtering at a boundary region of a macro block since arelatively big blocking exists at the boundary region of the macroblock(S13). This is explained in detail by referring to FIG. 2A, FIG.2B, and FIG. 3.

FIGS. 2A and 2B illustrate block diagrams indicating block locations inhorizontal and vertical directions according to an embodiment of thepresent invention, respectively, and FIG. 3 illustrates a diagram ofpixel locations of adjacent blocks according to an embodiment of thepresent invention, in which block locations in directions of horizontaland vertical axes are shown.

First of all, the step of determining the block strength St and afiltering state is explained as follows.

The step of determining the block strength St and a filtering state is aprocess for determining the H.26L system as standards, for which TML(test model long-term number 5) is in progress. In this case, the TMLcompresses a video by taking a 4×4 block as a reference.

The TML (test model long-term) moving picture compression system uses atechnique of removing spatial and temporal redundancy information froman encoder so as to transmit spatially & temporally compressedinformation and additional information required for decoding. Of course,in order to remove the redundancy information on a temporal domain, MPEG(moving picture expert group) and H.263 motion compensation systemaccording to the related art are used. The technique of removing theblocking using a basic mechanism of such a TML system is generallytreated by 4×4 block unit.

In this case, H.263 and H.26L are moving picture compression standardsproposed by ITU-T (international telecommunicationsunion-telecommunication).

The system for removing the blocking according to an embodiment of thepresent invention may follow the system of the related art. Yet, thefiltering state is determined by the following pre-treatment steps 1 to3 since the degree of the blocking differs in accordance with a codedblock pattern, a quantization step size, and a motion vector size.

[Step 1]

Strength St of each 4×4 block is initialized by ‘0’(St=0).

[Step 2]

If each of the 4×4 blocks is an intra-coded block or has a non-zerotransform coefficient, St=max(St,2).

[Step 3]

If a motion vector difference between respective motion vectors for thetwo blocks block1 and block2 in FIG. 2( a) and FIG. 2( b) is equal to orgreater than ‘1’, St=Max(St,1).

In this case, max(a,b) means a maximum value of ‘a’ and ‘b’. Namely,max(a,b) is a function for selecting a greater value from ‘a’ and ‘b’.

Moreover, after the strength St of each of the blocks has beendetermined, the loop filtering is carried out in accordance with thequantization size of each of the blocks.

For carrying out the loop filtering, the quantization sizes of the twoblocks block1 and block2 are as follows.

First case is that the quantization value of the block block1 is smallerthan ‘21’ and the strengths of the two blocks block1 and block2 are not‘0’. And, second case is that the quantization value of the block block1is greater than ‘20’ and the strength St of one of the two blocks block1and block2 is not ‘0’.

In this case, the filtering state is determined differently inaccordance with the quantization size because an adaptive treatment isperformed due to the dominating generation of blocking in case of agreat quantization size.

Therefore, when the conditions for the pre-treatment steps 1 to 3 aremet, the loop filtering process is carried out by the following manner.

First, pixel locations of two blocks block1 and block2 are shown in FIG.3. If the above-defined filtering state of each of the blocks issatisfied, values of pixels c, d, e, and f are found by the followingFormula 2-1(for calculating filter coefficient values to be used inFormula 2-5).

[Formula 2-1]r1=reg{St₁,QP,abs(b−c)},r2=reg{St₁,QP,abs(c−d)},r3=reg{max(St₁+1,St₂+1),QP,abs(d-e)},r4=reg{St₂,QP,abs(e−f)},r5=reg{St₂,QP,abs(f−g)}

From Formula 2-1, reg(.) is defined by the following Formula 2-2.

$\begin{matrix}{{{{reg}\left( {{St},{Q\; P},{Diff}} \right)} = \frac{256 \times {M({St})} \times D\; Q\;{P\left( {Q\; P} \right)}}{{{Diff} \times {Diff}} + {{M({St})} \times D\; Q\;{P\left( {Q\; P} \right)}}}},} & \left\lbrack {{Formula}\mspace{14mu} 2\text{-}2} \right\rbrack\end{matrix}$where Diff is a random value and ‘abs’ indicates a function of absolutevalue.

And, M(St) is defined by the following Formula 2-3.

{Formula 2-3}M(St=0)=1/6,M(St=1)=4/16,M(St=2)=8/16,M(St=3)=1

And, DQP(QP) is explained by referring to FIG. 4 as follows.

FIG. 4 illustrates a table of LQP values for determining DQP(QP) valuesaccording to an embodiment of the present invention. Namely, DQP(QP) isdefined by the following Formula 2-4, and can be found by substitutingthe LQP values shown in the table of FIG. 4. LQP is a quantization tabledefined in H.26L, and DQP defines a quantization error range inaccordance with the quantization table defined in H.26L as defined byFormula 2-4. Namely, the LQP values are “620”, “553”, “429”, . . . ,“24”, and “22” if the quantization variables QP are “0”, “1”, “2”, . . ., “30”, and “31”, respectively. Namely, as the quantization variable(0,1, 2, . . . , 30, 31) increases sequentially, the LQP value decreaseswith a predetermined value(620, 553, 492, . . . , 24, and 22).

$\begin{matrix}{{D\; Q\;{P\left( {Q\; P} \right)}} = \frac{676}{L\; Q\;{P\left( {Q\; P} \right)}}} & \left\lbrack {{Formula}\mspace{14mu} 2\text{-}4} \right\rbrack\end{matrix}$

Formula 1-11 is equivalent to Formula 2-2, and QP², K_(P)(or K_(N)), and{g(i,j)−g(i,j−1)} in Formula 1-11 are parameters equivalent to DQP(QP),M(St), and Diff in Formula 2-2, respectively.

Hence, the filtered values of the pixels c, d, e, and f are calculatedby the following Formula 2-5 using the defined parameters.

[Formula 2-5]c _(mod)={(512−r ₁ −r ₂)×c+r ₁ ×b+r ₂ ×d+256}>>9d _(mod)={(512−r ₂ −r ₃)×d+r ₂ ×c+r ₃ ×e+256}>>9e _(mod)={(512−r ₃ −r ₄)×c+r ₃ ×e+r ₄ ×f+256}>>9

f_(mod)={(512−r4−r₅)×f+r₄×e+r₅×g+256}>>9, where “>>” means a bit shift,and C_(mod), d_(mod), e_(mod), and f_(mod) mean values of filtering c˜fhaving the blocking shown in FIG. 3.

Meanwhile, the result from the above-explained processing brings aboutan effect of removing the blocking and ringing with satisfaction. Yet,micro blocking still exists in the boundary region of the macro blockconstituted by 16×16 block unit. In order to settle the micro blocking,an additional filtering, as shown in Formula 2-6, is carried out on thepixel lying at the boundary region of the macro block.d _(mod)=(c _(mod)+13×d _(mod)+2×e _(mod)+8)>>4

e_(mod)=(2×d_(mod)+13×e_(mod)+f_(mod)+8)>>4, if QP>16 and d and e aremacro block boundary pixels.

Accordingly, the present invention carries out a real-time processing ofremoving a blocking and a ringing phenomenon in accordance with a pixellocation, a coded block pattern, a quantization step size, and a motionvector size in a digital video appliance using the H.26L moving picturecompression technique, thereby enabling to improve a video quality of acompression image demanding a low bit rate or high-speed processing.

The foregoing embodiments are merely exemplary and are not to beconstrued as limiting the present invention. The present teachings canbe readily applied to other types of apparatuses. The description of thepresent invention is intended to be illustrative, and not to limit thescope of the claims. Many alternatives, modifications, and variationswill be apparent to those skilled in the art.

What is claimed is:
 1. A method of removing a blocking artifact usingquantization information in a filtering system, comprising: obtainingquantization information related to at least one of a first block and asecond block, the second block being adjacent to the first block;comparing the quantization information with a predefined constant valuethat remains fixed over more than one execution of a filtering processfor a plurality of frames to determine whether the quantizationinformation is greater than or less than or equal to the predefinedconstant value; and performing the filtering process based on a resultfrom the comparing step, the filtering process including, obtaining afiltered value of at least one pixel in the first block based on pixelvalues of at least two pixels in the first block, and obtaining afiltered value of at least one pixel in the second block based on pixelvalues of at least two pixels in the second block, wherein the at leasttwo pixels in the first block and the at least two pixels in the secondblock are successive pixels across a block boundary between the firstblock and the second block.